(1) Field of the Invention
The invention pertains generally to underwater cameras. More specifically, the invention relates to a closed-form solution to single underwater camera calibration using triple wavelength dispersion and its application to 3D reconstruction of a target object.
(2) Description of the Related Art
Computer vision has been a popular method to explore the underwater environment used by several groups of researchers such as biologists and geologists. It has many applications such as scientific exploration of the structure of the sea floor, and the monitoring of the undersea habitat. Hence, many researchers are interested in underwater computer vision. A fundamental and important topic in this area is underwater camera calibration which includes estimating the camera housing parameters. Camera calibration is a required step for recovering the 3D geometry of a scene using 2D images.
Examples of related work in this field include the following:
[1] A. Agrawal, S. Ramalingam, Y. Taguchi, and V. Chari. A theory of multi-layer flat refractive geometry. In CVPR, pages 3346-3353, 2012.
[2] A. N. Bashkatov and E. A. Genina. Water refractive index in dependence on temperature and wavelength: a simple approximation. In SPIE, pages 393-395, 2003.
[3] Y.-J. Chang and T. Chen. Multi-view 3d reconstruction for scenes under the refractive plane with known vertical direction. In ICCV, pages 351-358, 2011.
[4] V. Chari and P. Sturm. Multiple-view geometry of the refractive plane. In BMVC, London, UK, 2009.
[5] X. Chen and Y.-H. Yang. Two-view camera housing parameters calibration for multi-layer flat refractive interface. In CVPR, pages 524-531, 2014.
[6] R. Ferreira, J. Costeira, and J. A. Santos. Stereo reconstruction of a submerged scene. In IbPRIA, pages 102-109, 2005.
[7] A. Jordt-Sedlazeck and R. Koch. Calibration of housing parameters for underwater stereo-camera rigs. In BMVC, pages 118.1-118.11, 2011.
[8] A. Jordt-Sedlazeck and R. Koch. Refractive structure-from-motion on underwater images. In ICCV, pages 57-64, 2013.
[9] L. Kang, L. Wu, and Y.-H. Yang. Experimental study of the influence of refraction on underwater three-dimensional reconstruction using the svp camera model. Applied Optics, 51(31):7591-7603, 2012.
[10] L. Kang, L. Wu, and Y.-H. Yang. Two-view underwater structure and motion for cameras under flat refractive interfaces. In ECCV, pages 303-316, 2012.
[11] R. Kawahara, S. Nobuhara, and T. Matsuyama. A pixel-wise varifocal camera model for efficient forward projection and linear extrinsic calibration of underwater cameras with flat housings. In ICCV Workshop, pages 819-824, 2013.
[12] J.-M. Lavest, G. Rives, and J.-T. Lapreste. Underwater camera calibration. In ECCV, pages 654-668, 2000.
[13] O. Pizarro, R. Eustice, and H. Singh. Relative pose estimation for instrumented, calibrated imaging platforms. In Proc. of DICTA, pages 601-612, 2003.
[14] J. P. Queiroz-Neto, R. L. Carceroni, W. Barros, and M. F. M. Campos. Underwater stereo. In SIBGRAPI, pages 170-177, 2004.
[15] D. Scharstein and R. Szeliski. High-accuracy stereo depth maps using structured light. In CVPR, pages 195-202, 2003.
[16] T. Treibitz, Y. Y. Schechner, and H. Singh. Flat refractive geometry. In CVPR, 2008.
[17] T. Yau, M. Gong, and Y.-H. Yang. Underwater camera calibration using wavelength triangulation. In CVPR, 2013.
[18] Z. Zhang. A flexible new technique for camera calibration. IEEE Transactions on PAMI, 22(11):1330-1334, 2000.
Each of the above referenced documents are incorporated herein by reference.
Despite the remarkable success [18] for land-based camera systems, underwater camera calibration has not been addressed until recently. In a typical underwater camera system, the camera is placed inside a watertight housing, and views the scene through a flat piece of glass. As a result, the light that travels into the camera undergoes two refractions and its path is not a straight line, the result of which causes distortion in the captured images. The distortion depends on the scene depth and cannot be simply modeled as lens radial distortion [13]. Therefore, calibrating underwater cameras remains a challenging problem in the computer vision area.
In underwater computer vision, the refraction effect is sometimes ignored [14], or approximated by [6, 9, 12]. However, since the refraction effects are highly non-linear and depend on scene geometry, these methods usually produce errors. Treibitz et al. [16] show that the bundle of rays imaged by a perspective camera through a refractive interface does not correspond to a single-viewpoint (SVP) camera. They develop a technique to recover the distance from the camera center to the interface under the assumption that there is only one refraction. The method requires a planar checkerboard pattern with known distance from the camera. Another requirement which is not practical is to require the refractive interface parallel to the image plane.
Other problems and limitations are also apparent from the existing work. Chari and Sturm [4] derive a 12×12 refractive fundamental matrix analogous to the ordinary fundamental matrix. However, there is no demonstrated practical application disclosed in their paper. A 3D reconstruction method is proposed in [3] which models the refraction effect as a function of scene depth. One limitation is that it requires an Inertial Measurement Unit (IMU) to provide the vertical direction of each view. Another limitation is that all the cameras must share the same interface. A calibration method is presented in [7] which does not require a calibration target. It can account for two refractions by assuming that the glass thickness is known. One limitation is that its nonlinear optimization takes hours and the results of real data are not compared to the ground truth. Kang et al. [10] develop an optimization procedure with the limitation that the refractive interface is parallel to the image plane. Agrawal et al. [1] propose an efficient calibration method by showing that the flat refractive geometry corresponds to an axial camera. With this finding, all the refraction parameters can be computed by solving a set of linear equations. Non-linear optimization is still required to refine the results. Nevertheless, the method assumes the 3D geometry of the calibration target to be known. Therefore, a checkerboard pattern is typically used which may not be practical when the cameras are deployed deep undersea. More recently, Yau et al. [17] extend the work of [1] by accounting for the dispersion of light which improves the calibration accuracy. One limitation is that it requires a heavy custom-built submersible light box which weighs over 60 lbs. The method proposed in [8] uses a “virtual camera” error function where each 3D point is projected using an imaginary perspective camera (the “virtual camera”), and an iterative nonlinear optimization is applied to minimize the reprojection error. It is claimed that high accuracy is achieved in calibrating the refraction parameters. However, the results of real experiments are not evaluated against the ground truth. A virtual camera model is presented in [11] to model refraction as a pinhole camera with a specific focal length for each image pixel. The model can be used for estimating the camera pose when the housing parameters are provided. A new method is presented in [5] to calibrate the housing parameters with a limitation that the camera pose is provided.